1
Foundations of Numerical Differentiation
MATH007 Lesson 4
00:00
Numerical differentiation marks the high-stakes transition from the infinite smoothness of calculus to the discrete, finite world of digital computation. We trade the infinitesimal limit for a measurable step-size $h$. While the theoretical derivative of $f$ at $x_0$ is defined as $$f'(x_0) = \lim_{h \to 0} \frac{f(x_0 + h) - f(x_0)}{h}$$, computer systems cannot calculate a limit directly. Instead, we use finite difference formulas, incurring a quantifiable penalty known as truncation error.

1. The Geometry of the Derivative

To approximate $f'(x_0)$, we look at neighboring points. Depending on our choice of direction, we derive two primary formulas:

  • Forward-difference formula: Used if $h > 0$. It looks ahead to $x_0 + h$.
  • Backward-difference formula: Used if $h < 0$. It looks back to $x_0 + h$ (where $h$ is negative).

In real-world engineering, such as calculating the arc length of a curved trajectory, we often rely on these approximations: $$L = \int_{0}^{48} \sqrt{1 + (f'(x))^2} dx = \int_{0}^{48} \sqrt{1 + (\cos x)^2} dx$$ If $f(x)$ is only known at discrete sensor points, numerical differentiation is the only path forward.

2. Mathematical Derivation via Interpolation

To approximate $f'(x_0)$, suppose first that $x_0 \in (a, b)$, where $f \in C^2[a, b]$, and that $x_1 = x_0 + h$. We construct the first Lagrange polynomial $P_{0,1}(x)$ determined by $x_0$ and $x_1$:

Step 1: Interpolant Construction
$f(x) = P_{0,1}(x) + \frac{(x - x_0)(x - x_1)}{2!} f''(\xi(x))$
Step 2: Differentiation
Differentiating both sides and evaluating at $x = x_0$ yields the fundamental relation: $$f'(x_0) = \frac{f(x_0 + h) - f(x_0)}{h} - \frac{h}{2} f''(\xi)$$

3. The Error Term and Convergence

The term $-\frac{h}{2} f''(\xi)$ is our truncation error. This formula demonstrates that the accuracy is $O(h)$, meaning if you halve the step size $h$, you roughly halve the error. However, we must be cautious: while smaller $h$ reduces truncation error, it eventually increases round-off error due to the subtraction of nearly identical numbers in the numerator.

🎯 Core Principle: The Finite Difference
Numerical differentiation replaces the limit with a finite chord. The quality of our approximation depends strictly on the step size $h$ and the smoothness (second derivative) of the function.
$f'(x_0) \approx \frac{f(x_0+h) - f(x_0)}{h}$ with Error Bound $\frac{h}{2} \max|f''(\xi)|$